There is an increase of communication applications for transmitting video data and audio data from the transmission side in the form of IP (Internet Protocol) packets and for reconstructing and reproducing the video data and audio data from the IP packets on the reception side. Examples of such applications include applications for IP phone (VoIP), TV conference, video distribution of movies and live performances and the like on the internet.
Usually, in the application that handles real-time data such as video, audio, etc., when a packet is lost, a concealment technique, that tries to recover the lost packet from among the packets that arrived before the packet was lost and from among the packets that arrived after the packet was lost, has been used. This makes it possible to conceal the effect of packet loss. This one-off packet loss that is not continuous will be called a single loss.
However, once multiple packets from the transmission side have been continuously lost due to network disorder or any other reason, it is difficult to recover the packets that have lost, from the packets having arrived before and after the loss occurred, so that there occur cases in which video and/or audio is interrupted. Such a continuous packet loss will be called a continuous loss.
In this way, in the application that sends and receives video and audio and reproduces them, continuous loss of packets gives rise to a serious problem in maintaining the quality of video and audio.
In order to avoid the influence from the continuous loss of packets, use of a technology called an interleave method whereby the order of data is rearranged has been investigated. For example, there is a technique disclosed by Yi J. Liang, J. G Apostolopoulos, and B Girod, in “Analysis of packet Loss for Compressed video: Does Burst-Length Matter?”, IEEE International Conference on Acoustics, Speech, and Digital Processing, IC ASSP2003, April 2003 (which will be referred to hereinbelow as document 1).
FIG. 1 shows how data is rearranged by the interleave method. State 1 in the same figure shows the stage before rearrangement. In state 1, there are, in total, 16 pieces of continuous data, A1˜A4, B1˜B4, C1˜C4 and D1˜D4. If no interleaving process is performed, these 16 pieces of data are transmitted as they are in state 1, or in the order of A1->A2->A3->A4->B1->B2-> . . . ->D3->D4. In contrast, when an interleaving process is carried out in order to enhance resistance against continuous loss, the data is arranged in the order shown in state 2, or the data is transmitted in the order of A1->B1->C1->D1->A2->B2-> . . . ->C4->D4.
Now, consider a case where data A2, B2, C2 and D2 in state 2 are continuously lost. In this case, on the reception side, data A2, B2, C2 and D2 have dropped out as shown in state 3. The hatching in the drawing shows the continuous loss of data. The data indicated by the broken line represents the lost data. On the reception side, a process of recovering the original order of data from the data that was permuted on the transmission side is executed. By this process, the data shown in state 3 is reordered into state 4. As a result, the continuously lost data A2, B2, C2 and D2 will be scattered in the time-axis direction.
On the other hand, when no interleaving process is adopted, once continuous data loss of data occurs, data B1˜B4 are lost as shown in state 5. In this case, data B1˜B4 have been lost continuously, so that it is difficult to make recovery, hence video and/or audio will be interrupted.
In contrast, when the interleaving process is adopted, the continuous loss can be made to look like single losses as shown in state 4. Accordingly, it is possible to reproduce the video and audio by restoring the lost data.
FIG. 2A shows the arrangement order of transmitted data after an interleaving process is performed when the interleave length is “4”. FIG. 2B shows the arrangement order of transmitted data after interleaving is performed when the interleave length is “6”. When the interleave length is “4”, the reception side receives three pieces of data B1, C1 and D1 from the time when data A1 is received until the time when data A2, that is to be reproduced after A1, is received. Accordingly, reproduction is delayed by the amount of time required for receiving three pieces of data. When the interleave length is “6”, five pieces of data are received from the time when data A1 was received until the time when data A2 is received. Accordingly, reproduction is delayed by the amount of time required for receiving five pieces of data. In the interleaving scheme, the longer the interleave length is, the more the resistance to continuous loss is enhanced, though the delay of data reproduction becomes greater.
For an application that handles real-time data such as video and audio, it is preferable that delay be short, hence it is important to select the interleave length appropriately. Concerning determination of the interleave length, document 1 and “Model-Based Delay-Distortion Optimization for Video Stream Using Packet Interleaving” IEEE 36th Asilomar Conference on Signals, Systems and Computers, November 2002 (which will be referred to hereinbelow as document 2) by Yi J. Liang, J. G Apostolopouos, and B. Girod, propose methods of calculating the interleave length based on the permissible amount of delay at the time of reproduction, which has been set up previously.
Further, Japanese Patent Application Laid-open 2004-32283 (which will be referred to hereinbelow as document 3) discloses a technology of changing the interleave length in accordance with the reception condition. In document 3, the discarded ratio of packets is calculated based on the number of packets that have been discarded through the transmission path while the transmission throughput is calculated based on the time taken from the first packet transmission to the last packet transmission and the total amount of data. Then, based on the packet discarded ratio and the transmission throughput, the payload data size and the interleave length are determined. There is also a reference in document 3 in which the interleave length is made longer when, for example the packet discarded ratio is large.